1. Field of the Invention
The present invention relates to the field of computer programming, and deals more particularly with a method, system, and computer program product for including information items having various media formats (such as text, image, audio, etc.) in a user interface that is specified in, and will be rendered from, a markup language document (such as an Extensible Markup Language, or xe2x80x9cXMLxe2x80x9d, document).
2. Reservation of Copyright
A portion of the disclosure of this patent document contains material to which a claim of copyright protection is made. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all other rights whatsoever.
3. Description of the Related Art
The advanced user interfaces used by state-of-the-art applications often provide information using a wide variety of media, both for the user interface (UI) and for descriptive information items associated with the interface. The types of descriptive information items in common use today include tool tips; audio or visual explanations for an element presented via the UI, where these explanations may be selectable by a user; help text; hints, including xe2x80x9chover helpxe2x80x9d information that is provided when a user allows the cursor of a pointing device to hover over an element of a displayed UI; etc. The media types in which these descriptive items are provided may include text, video clips, graphic images, audio or speech, etc. Because these kinds of media are typically human-language specific, they are by convention stored in files which are separate from the files containing the software code of the application and/or the code from which the UI is created. This separate storage facilitates translating the stored information into different natural languages, sometimes referred to as xe2x80x9cinternationalizationxe2x80x9d of the information, without requiring changes or recompilation of the executable software. xe2x80x9cResource filesxe2x80x9d is a term commonly used to refer to these separately-stored media files, referring to the fact that the stored media information is used as a resource by an application program. An application typically includes a particular resource file by specifying a reference to the file system location where the resource file is stored (e.g. by providing the file system name to a compiler and/or linker utility). If multiple resource files are needed by the application, then multiple of these references are used. Resource files may be shared among multiple applications. These concepts are well known in the art.
Techniques for referencing media resource files from markup language documents are also known in the art. In the Hypertext Markup Language (xe2x80x9cHTMLxe2x80x9d) notation, for example, the HTML syntax which specifies a Web page may include a reference to a separately-stored image, or video clip, or other type of media resource. Typically, the Web developer specifies an explicit reference to the separately-stored media resource using its Uniform Resource Locator (URL) or Uniform Resource Identifier (URI). The developer of an Extensible Markup Language (xe2x80x9cXMLxe2x80x9d) document may also specify references to separately-stored media resources, which is also typically done by specifying a URL or URI reference. HTML and XML are both tag languages, where a tag language uses specially-designated constructs referred to as xe2x80x9ctagsxe2x80x9d to delimit (or xe2x80x9cmark upxe2x80x9d) information. In the general case, a tag is a keyword that identifies what the data is which is associated with the tag, and is typically composed of a character string enclosed in special characters. xe2x80x9cSpecial charactersxe2x80x9d means characters other than letters and numbers, which are defined and reserved for use with tags. Special characters are used so that a parser processing the data stream will recognize that this a tag. A tag is normally inserted preceding its associated data: a corresponding tag may also be inserted following the data, to clearly identify where that data ends. As an example of using tags, the syntax xe2x80x9c less than email greater than xe2x80x9d could be used as a tag to indicate that the character string appearing in the data stream after this tag is to treated as an e-mail address; the syntax xe2x80x9c less than /email greater than xe2x80x9d would then be inserted after the character string, to delimit where the e-mail character string ends.
While HTML uses a predefined set of tags, the syntax of XML is extensible and allows document developers to define their own tags which may have application-specific semantics. Because of this extensibility, XML documents may be used to specify many different types of information, for use in a virtually unlimited number of contexts. (A number of XML derivative notations have been defined, and continue to be defined, for particular purposes. xe2x80x9cVoiceXMLxe2x80x9d is an example of one such derivative. References herein to xe2x80x9cXMLxe2x80x9d are intended to include XML derivatives and semantically similar notations such as derivatives of the Standard Generalized Markup Language, or xe2x80x9cSGMLxe2x80x9d, from which XML was derived. Refer to ISO 8879, xe2x80x9cStandard Generalized Markup Language (SGML)xe2x80x9d, (1986) for more information on SGML. Refer to xe2x80x9cExtensible Markup Language (XML), W3C Recommendation 10-Feb.-1998xe2x80x9d which is available on the World Wide Web from the World Wide Web Consortium, or xe2x80x9cW3Cxe2x80x9d, for more information on XML.)
The present invention is concerned with XML documents which are used to specify a user interface. In this UI context, XML tags may be defined which specify media resources that may be used to provide the descriptive items described above, such as help text, hints, etc. For example, if a UI has an element such as a text entry field requesting a user to enter his or her last name, this element may have an associated hint which reminds the user that this is the surname. The element may also have associated help information which provides more detailed information, such as guidance on any syntactic data entry requirements for the surname, references to other places where the surname value may have been previously entered, etc. This hint and/or help information may be rendered using text, audio, etc. A UI designer may in some cases choose to provide multiple media types for one or more of the descriptive information items associated with a particular UI element. In the surname example, the user may be given a choice of receiving the hint in audio or text format, in which case the UI XML document must specify or reference both types of media resource files. FIG. 1 provides an example of prior-art XML syntax 100 that may be used for specifying UI information for a single UI field. In this example, a  less than STRING greater than  tag pair 105, 160 represents information for a last name, as indicated at 115 for the NAME attribute 110. The  less than HINT greater than  tag pair 120, 130 brackets a textual hint 125. The  less than HELP greater than  tag pair 140, 150 brackets textual help 145 which is enclosed within  less than META-TEXT greater than  142 and  less than /META-TEXT greater than  147 tags. (In this example, a shortcut notation for the text within the  less than HINT greater than  tag has been used, where this hint text has not been enclosed within  less than META-TEXT greater than  tags.)
The techniques which exist in the prior art have a number of drawbacks, however. As will be obvious, as a UI XML document provides resource tags for more types of descriptive items, and/or more different media types for each descriptive item, the tagging in the source document becomes more verbose and cluttered, as well as more complex. The data content within the UI document may become obscured due to these multiple resource tags. In addition, a great deal of time can be expended when the developer of the UI XML document must explicitly and manually specify each type of descriptive information item to be provided for the UI content, and each media type to be provided for those items. In complex projects where one person or team develops the UI content and a different person or team develops the descriptive items, there may be a significant amount of overhead effort expended to coordinate sharing of files between these persons or teams during the development and testing phases, and to perform an integration of the final results. Furthermore, once such information has been statically specified for a particular UI XML document, it can be a time-consuming and error-prone process to revise that document such that it uses different descriptive items and/or different media types. The resource capabilities which are currently available in the art (such as in the Windows operating system and Java programming language) are rather limited. There is no capability for cross-referencing resources within a file to allow multiple use of a resource, for example, and very little structured support for resources. (For example, Windows allows definition of a dialog box, but this dialog box definition cannot reference other resources such as strings or other types of media in the resource file.)
Accordingly, what is needed is an improved technique for including descriptive information items, and the various media resources used for those items, in the markup language document from which a user interface is to be rendered.
An object of the present invention is to provide an improved technique for including descriptive information items, and the various media resources used for those items, in a user interface.
Another object of the present invention is to provide this technique by using one markup language document for the UI data content, and a separate markup language document for specifying the descriptive items and media resources to be used along with the content of that UI document.
Still another object of the present invention is to provide a technique which enables dynamically altering the descriptive items and media resources to be used for a UI document, without requiring change to the UI document itself.
Yet another object of the present invention is to provide a technique whereby the descriptive information items and media resources to be used for a UI document are specified in an MRI document, where the UI document and the MRI document may be developed independently.
Another object of the present invention is to provide a technique for efficiently reusing specifications of descriptive information items and media resources.
A further object of the present invention is to provide this reuse of specifications among multiple UI documents.
Other objects and advantages of the present invention will be set forth in part in the description and in the drawings which follow and, in part, will be obvious from the description or may be learned by practice of the invention.
To achieve the foregoing objects, and in accordance with the purpose of the invention as broadly described herein, the present invention provides a computer program product, a system, and a method for flexibly including descriptive items and media types in a user interface. This technique comprises: providing a UI document, the UI document comprising a specification of one or more UI elements to be rendered to a user; providing a machine readable information (MRI) document, the MRI document comprising one or more aggregations of MRI elements, each of the aggregations specifying one or more descriptive items in one or more media formats, and wherein selected ones of the UI elements reference selected ones of the aggregations of MRI elements; applying the selected ones of the aggregations to expand the references by the selected ones of the UI elements, thereby creating a media-enhanced UI document; and rendering the media-enhanced UI document.
The MRI elements may comprise one or more of: media specifications, references to include one or more of the media specifications, descriptive media containers, and references to include one of more of the descriptive media containers. The media specifications may comprise one or more of: a text format, a video format, an audio format, and an image format. The technique may further comprise one or more exclusion statements to exclude particular ones of the descriptive media containers, and the applying may further comprise omitting the particular ones when expanding the references to create the media-enhanced UI document.
The descriptive media containers may comprise one or more of: a caption container, a hint container, a tip container, and a help container. In this case, the UI document may further comprise one or more exclusion statements to exclude particular media types within particular ones of the descriptive media containers, and the applying may further comprise omitting the particular media types when expanding the references to create the media-enhanced UI document.
The applying may further comprise creating a hash table for quick and efficient retrieval of the MRI elements for use by the rendering.
The UI document and the MRI document may be specified in an Extensible Markup Language (XML) derivative., which may be an abstract UI markup language (AUIML).
An alternate version of the MRI document may be used by the applying, thereby altering results of the rendering without requiring change to the UI document. The technique may further comprise: providing second UI document, this second UI document comprising second UI elements which reference particular ones of the aggregations of MRI elements; applying the particular ones of the aggregations to expand the second UI elements, thereby creating a media-enhanced second UI document by reusing the particular ones of the aggregations from the MRI document; and rendering the media-enhanced second UI document.
The present invention will now be described with reference to the following drawings, in which like reference numbers denote the same element throughout.